Field of the Invention
The disclosed technology relates to integrated circuits for an imaging system which has an array of optical sensors and an array of optical filters, and to corresponding systems and methods, and computer programs, and more particularly to hyperspectral imaging (HSI) systems.
Description of the Related Technology
Operation of known Hyperspectral imaging systems:
Hyperspectral imaging refers to the imaging technique of collecting and processing information from across the electromagnetic spectrum. Whereas the human eye only can see visible light, a hyperspectral imaging system can see visible light as well as from the ultraviolet to infrared. Hyperspectral sensors thus look at objects using a larger portion of the electromagnetic spectrum, as has been described at: http://en.wikipedia.org/wiki/Hyperspectral_imaging.
Certain objects leave unique ‘fingerprints’ across this portion of the electromagnetic spectrum. These ‘fingerprints’ are known as spectral signatures and enable identification of the materials that make up a scanned object. The hyperspectral capabilities of such imaging system enable to recognize different types of objects, all of which may appear as the same color to the human eye.
Whereas multispectral imaging deals with several images at discrete and somewhat narrow bands, hyperspectral imaging deals with imaging narrow spectral bands over a contiguous spectral range. It can produce the spectra for all pixels in the scene. While a sensor with 20 discrete bands covering the VIS, NIR, SWIR, MWIR, and LWIR would be considered multispectral, another sensor with also 20 bands would be considered hyperspectral when it covers the range from 500 to 700 nm with 20 10-nm wide bands.
Hyperspectral sensors collect information as a set of ‘images’. Each image represents a range of the electromagnetic spectrum and is also known as a spectral band. These images' each have two spatial dimensions and if images of a series of different spectral bands are effectively stacked to form a cube, then the third dimension can be a spectral dimension. Such a three dimensional hyperspectral cube is a useful representation for further image processing and analysis. The precision of these sensors is typically measured in spectral resolution, which is the width of each band of the spectrum that is captured. If the scanner picks up on a large number of fairly narrow frequency bands, it is possible to identify objects even if the objects are only captured in a handful of pixels. However, spatial resolution is a factor in addition to spectral resolution. If the pixels are too large, then multiple objects are captured in the same pixel and become difficult to identify. If the pixels are too small, then the energy captured by each sensor-cell is low, and the decreased signal-to-noise ratio reduces the reliability of measured features.
Current hyperspectral cameras produce a hyperspectral datacube or image cube, consisting of a stack of 2D images in the x-y plane of the scene in which each image of the stack contains information from a different frequency or spectral band. The spectral range that is captured is not limited to visual light, but can also span infra red (IR) and/or ultra violet (UV). The 3D Image Cube is captured by a hyperspectral imager, using a sensor that is inherently a 2D sensor. Therefore some form of scanning needs to be used, as is shown in FIG. 1 which shows a perspective representation of a cube with the spectral dimension extending vertically, and four views a) to d) of slices of the cube as follows:
Topview (a) shows the scene that needs to be captured. Left sideview (b) shows a vertical slice from the cube, representing an image obtained by a line scanner: all spectral bands are captured for one spatial line of the scene resulting in a 1D view. Line scanners or pushbroom systems thus capture a single line of the 2D scene in all spectral bands in parallel. To cover all spatial pixels of the scene, this type of system then scans different lines over time, for example by relative movement of the scanner and the scene.
Right sideview (c) shows a horizontal slice showing an image obtained by a starer: the complete 2D scene is captured in one spectral band. Starers or staring systems capture the complete scene in a single spectral band at a time with a 2D array of sensors and scan over different spectral bands in order to produce the 3D hyperspectral image cube. Bottom view (d) shows a sloping or diagonal slice through the cube, representing an image obtained by a hybrid line scanner/starer: the complete 2D scene is captured, but every spatial line is at a different height of the cube and so is a different spectral band. In this case a complete spatial image is acquired, but with every line at a different spectral band. In a single frame different spectral bands are then captured for different spatial lines. To capture the complete 3D image cube, with all spectral bands for all spatial lines, a combined spatial/spectral scanning is still needed, for example by relative motion between the scene and the 2D sensor array.
Construction of Known Hyperspectral Imaging Systems:
Hyperspectral imaging systems or cameras can consist of different discrete components, e.g. the optical sub-system for receiving the incoming electromagnetic spectrum, the spectral unit for creating the different bands within the received spectrum and the image sensor array for detecting the different bands. The optical sub-system can consist of a single or a combination of different lenses, apertures and/or slits. The spectral unit can consist of one or more prisms, gratings, optical filters, acousto-optical tunable filters, liquid crystal tunable filters etc or a combination of these.
A primary advantage of hyperspectral imaging is that, because an entire spectrum is acquired at each point, the operator needs no prior knowledge of the sample, and post-processing allows all available information from the dataset to be mined. The primary disadvantages are cost and complexity. Fast computers, sensitive detectors, and large data storage capacities are needed for analyzing hyperspectral data. Significant data storage capacity is necessary since hyperspectral cubes are large multi-dimensional datasets, potentially exceeding hundreds of megabytes. All of these factors greatly increase the cost of acquiring and processing hyperspectral data.
State-of-the-art hyperspectral imagers are therefore either research instruments as they are too slow and too expensive or either designed for a dedicated industrial application thereby lacking flexibility.